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CHAPTER 11 


TESTING AND DESIGN’ VERIFICATION 


11.1 GENERAL 


Acceptance testing and design verification are the final steps in acquiring a system, sub- 
system, or unit of hardware. The major objectives of the acceptance test and design 
verification program are to ensure that (1) system overall performance is of maximum 
benefit to the Government, (2) installation and equipment documentation is available for 
maintenance purposes, and (3) contractual obligations have been fulfilled. Acceptance 
testing and design verification are just as much a part of the equipment installation as 

is the acquisition of the equipment itself. The successful completion of the planned 
acceptance tests and design verification program signifies that the items of concern 

are ready for operational use. 


Periodic system tests are performed to determine the effectiveness of the equipments 
as installed. These tests also help to evaluate equipment maintenance and determine 
when equipment updating is required. 


11.2 ACCEPTANCE TESTS 


Acceptance tests are divided into two major groups: (1) those required for procure- 
ment and checkout of a new major system and (2) those required for subsystem and 
unit testing upon modernizing or increasing the capabilities of a communications 
station with "'off-the-shelf" or production-run equipments. 


11.2.1 Major System Testing 


Tests required for the acceptance ofa new major system start with the initial system design 
and continue throughout installation until final acceptance. Accordingly, the tests are 
divided into categories I, Il and II test phases, which are described briefly as follows: 


a. Category I Testing. This testing is conducted during system development and is 
concerned with individual components. It allows for redesign and refinement of the 
product. 


b.. Category II Testing. This testing is accomplished to validate system design and 
consists of testing and evaluating subsystems through the mating process that pro- 
gresses into a complete system. This testing is accomplished for major system pro- 
curement where it is necessary to prove system compatibility and to expose any re- 
quirements for system redesign. It is a joint effort by Government and contractor 
to allow continued development of the system. 


c. Category III Testing. This testing is performed to prove system maintainability, 


operability, and supportability within the planned skill levels of the assigned personnel, 
To allow realistic evaluation, category III testing is accomplished by user personnel. 
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The plan for accomplishing categories I, II and II testing is developed as a part of the 
total system acquisition package. Testing is done under the direction of the system proj- 
ect office assisted by field activity personnel. The system sponsor may participate in 
the testing to assure himself of system performance. Upon successful completion of 
category III testing, the system is ready for operational use. 


11.2.2 Subsystem and Unit Acceptance Testing 


Subsystem and unit acceptance testing must be accomplished for all newly installed 
units or subsystems to determine their ability to operate properly as an integrated 

part of the overall operational system. Whenever a unit of hardware or subsystem is 
installed an acceptance test program must be conducted. The successful completion 

of this acceptance test program will constitute acceptance for operational use by both 
the Government and operating personnel. The acceptance test program may range from 
simple meter checks to complex operational tests depending upon the type and quantity 
of equipments installed. When an acceptance test program for hardware installations 
is being planned, the major considerations are as follows: 


a. User requirements. 

b. Verification of system capabilities (demonstration of unit or system). 
c. Duration of testing. 

d. Manpower, test equipment, and cost. 

e. Procedures to be followed if minimum requirements are not met. 


Since the individual equipments involved are usually items that have been previously 
subjected to acceptance tests for the Navy, it is necessary to test the equipment only 
for its intended operational purpose. Re-measuring and rechecking of fixed charac- 
teristics defined by manufacturing specifications, or electronic realignment of in- 
dividual equipments of a system is not normally required at the time of installation. 
However, checks of system characteristics that are controlled by tung, peaking or 
otherwise adjusting the newly installed subsystem or unit must be included as part of 
the acceptance testing. 


11.2.3 Acceptance Test Plan 


The acceptance test plan is a written document which prescribes an orderly procedure 
to accomplish the tests and defines the extent of the testing. The test plan consists of 
two basic parts, the test directive and the test procedure. The salient factors to be 
considered when writing a test plan are outlined below: 


a. Test Directive 
(1) Introduction. Give general or background information leading up to publi- 
cation of the test directive authority for and priority of the test program: mention any 


previous testing and related projects to be completed. 


(2) Purpose of the Test. State briefly the reason for the test and include a gen- 
eralized statement of the objectives. 
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(3) Scope of the Test. Specify detailed objectives that must be met to fulfill the 
purpose of the test. 


(4) Method of Conducting the Test. Describe the manner and sequence in which 
subsystem or system tests are to be conducted. Include prerequisites for the condition 
of the system. 


(5) Security. Classify, as necessary, equipment, tests, test results, documen- 
tation and other material related to the test program. 


(6) Test Environment and Test Items. Describe the items to be tested and the 
environment(s) in which tests will be conducted. 


(7) Organizations and Responsibilities. Provide an organizational breakdown 
showing participating units, functions, and specific responsibilities necessary for a 
timely and successful test program. 


(8) Resources. Specify the resources required and how and where they will be 
used. Include equipment and facility requirements according to time schedules, loca- 
tions, and source of support. 


(9) Instrumentation and Data Collection. Delineate the method of data collection, 
specifying whether automatic or manual, and indicate where it will be performed. In- 
clude requirements for special instrumentation needs and data handling. Refer to 
standard data collection forms, if applicable, or forms specially prepared for this 
purpose. 


(10) Date reduction. Indicate methods of processing and the form in which the 
date will be presented; specify requirements for equipments such as digital computers, 
oscillograph scanners, etc., needed for data reduction; express in hours the computer 
time required, and list requirements for any special computer programs. 


(11) Schedules. Arrange test schedules into a logical sequence according to ob- 
jectives, milestones, and target dates. Schedule realistic and coordinated test dates. 


(12) Test Analysis and Reports. Indicate the person(s) who will analyze the test 
results; also state the type and frequency of reports required and to whom they will 
be sent.. 


b. Test Procedure 


(1) Installation and Evaluation. Prescribe checks of installation plans to ensure 
proper workmanship. Include checks for placement in accordance with the plan, ade- 
quate foundations and equipment mounting, proper cable and wiring procedures, and 
correct wire termination. 

(2) Unit Tests. In general, individual equipment tests should be made in accor- 
dance with the procedures prescribed in the equipment instruction books (NAVSHIPS, 
TOs, TMs, etc.) for initial setup. These tests are not intended to duplicate, sub- 
stitute or modify any in-plant acceptance tests required by procurement contracts. 
The tests are confined to the equipment itself and may require the use of either built- 
in or external test equipment. Test results showing a level of performance below 
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standard shall be regarded as deficient. Antennas will require the equivalent of 
equipment tests. Refer to NAVELEX 0101,104 — "HF Radio Antenna Systems" for test 
parameters. Testing of microwave, tropospheric scatter and other systems will be parts 
of criteria books yet to be issued. 


(3) Operational Test. Prescribe the performance testing of the equipments as a 
complete entity over a specific time period for the purpose of demonstrating the ability 
to fulfill operational requirements. Performance of this test may require the introduc- 
tion of a wide range of simulated operational situations to demonstrate fully the compat- 
ibility and capabilities of the subsystem. In general, system tests should include all 
of the equipments and interconnections normally comprising a system, from the site 
input to the site output. The intent of system tests is to verify the compatibility of 
equipments comprising a system. If possible, include equipments from each station site 
in overall system tests. 


(4) Design Verification. Prescribe the types and amount of design verification. 
The major points fo be included are: 


a. that equipments are provided in accordance with the installation plans. 


b. that ''as-built'’ drawings and manuals, spare parts, special tools, and 
special test equipment are on hand. 


c. that adequate on-the-job training is provided for operator and maintenance 
personnel. 


11.3 PERIODIC SYSTEM TESTS 


In the past, the Naval Electronic Systems Command has made periodic equipment 
inspections within the communications stations. These inspections were found to be of 
marginal utility in improving the communications capabilities of the station inspected. 
Consequently, the equipment-by-equipment inspection is being replaced by a communi- 
cations capability test. Such tests will not only be conducted biennially at each communi- 
cations station, but also when major changes in equipment have been made. 


The communications capability test measures the degree of distortion introduced into 
a discrete information sample while it is being processed at a single station. This 
test covers all the equipment from the introduction device to the HF propagation path 
and from the HF propagation path to the end instrument. The result of the test is an 
accurate error count of the data sample after it has passed through the receive or 
transmit capability of the station. This capability test is performed for both analog 
and digital circuits. The test analysis is based on the required circuit speed (data 
baud rate) and upon the amount of distortion introduced into the sample as it flows 
through the station. 


To perform this test, NAVELEX provides a team of experts equipped with special 

test equipment. The test is designed to disrupt as little as possible the normal 
communications actions of the station. For example, the equipment is not disabled, 
and while individual circuits are being tested the communications normally conducted 
by that circuit are shifted to standby equipment. As the test methods are perfected, 

it is envisioned that test equipment may be installed within a station to provide a self- 
testing capability that can be used after preventive maintenance action and when system 
degradation is suspected. 
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11.3.1 Test Procedure 


The test procedure developed to date is restricted to digital and teletype circuits. A 
discrete 20, 000-baud test message is processed through both the send and receive sides 
of each circuit of the communications station, The test message is produced by a 
pattern generator that can be set for various precise baud speed rates. The 20,000 - 
baud message is sent through a system at the rated circuit capacity. Then the message 
speed is increased until 50 percent above the rated circuit capacity is attained. 


The send side of the circuit is tested by introducing the test message at the DC patch- 

board nearest the send device and measuring the output with a test stand located at the 
transmitter site (figure 11-1). The test stand employed at the transmitter site consists 

of a radio receiver accurately controlled with a frequency standard, a demodulator, and 

an error count detector that is set to evaluate accurately the received signal for distortion 
and total error count. The receive side of the circuit is tested in a similar manner; the 
test message is inserted at the receiver site, and the results are measured at the DC patch- 
board nearest the end equipment (figure 11-2). The transmitter used to couple the test 
message to the receive antenna is also carefully controlled in frequency and stability. 


11.3.2 Test Results 


Test data indicate that the communications capability test can be used effectively to 
evaluate the circuits of a station. Figure 11-3 is a graph of the test results of a 
3-kHz bandwidth receive system employing a VFCT terminal. The history of the test 
follows. 


A communications station was asked to allow the communications capability test to be 
made on a receive system that was considered to be in excellent condition both from 
maintenance and operational standpoints. The test results, curve 1 of figure 11-3, 
show that only 82 percent of the digital channels of the VFCT system would pass 
errorless information at the 75-baud rate. The test team asked the maintenance per- 
sonnel to conduct a routine preventive maintenance check of the system and to per- 
form every step of the procedure as prescribed by the manufacturer's maintenance 
manual for each equipment in the system. Upon completion of the maintenance actions, 
the communications capability test was again conducted, and curve number 2 of 
figure 11-3 was obtained. This curve indicates that all of the channels of the VFCT 
equipment were then capable of passing errorless traffic up to the 109-baud rate. 


The communications capability tests have shown that frequency control and frequency 
accuracy of the transmitter and the VFCT equipment are major factors in passing digital 
and teletype traffic without introducing errors. The effect of frequency deviation of the 
transmitter during transmission of single-channel, frequency-shift-keying, 100-words 
per- minute teletype, using a frequency shift of +425 Hz, is shown by figure 11-4. 
Frequency control and accuracy are even more critical in the VFCT equipment. 

Figure 11-5 shows the test results for an individual channel of the VFCT equipment 
employing a +42. 5-Hz center-frequency keying shift. 
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Figure 11-1. Typical Send Circuit Test 
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Figure 11-3. 3-kHz VFCT Receive System Test Results 


The test results are being used to equate teletype circuit distortion to teletype charac- 
ter error rate. The graph of figure 11-6 has been developed through empirical test 
data, and it is also compatible with the DCS criteria for satisfactory teletype trans- 
mission of not more than one error in 10, 000 transmitted characters. This graph can 
be used as an indication of circuit error rate as derived from the readings of teletype 
distortion analyzing equipment. 


11.3.3 Test Analysis 


Analysis of the test data is based on the error count observed for a 20, 000-baud test 
message transmitted at various baud rates. Each circuit is then rated as follows: 


Good - No errors observed at the 100-baud rate. 


Marginal - Two errors or less at a baud rate of 100 and none at a baud rate of 75. 
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Figure 11-4. Single Channel FSK (+425 Hz) Transmission Test Data 
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Figure 11-5. VFCT Channel Frequency Deviation Test Data 
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Figure 11-6. Teletype Distortion Rate Versus Error Rate 


Satisfactory - No errors observed at the 75-baud rate. 
Inoperable - More than one error at the 75-baud rate. 


The limit of one error in a 20, 000-baud test message is based upon the DCA criter 
for satisfactory medium speed digital transmission of not more than one error in 
100, 000 transmitted bits. 


Communications capability tests reveal that high error rates often exist without the 
knowledge of station personnel. The analysis of an overseas station subjected to this 
test is shown in table 11-1 and figure 11-7. Of the circuits tested, 16.2 percent were 
rated inoperable. Great variations were found in the capability of the VFCT equip- 
ment to pass errorless traffic. The degree of variation between VFCT systems is 
indicated by figures 11-8 and 11-9. 
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Figure 11-7. Circuit Evaluation 
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Figure 11-8. VFCT Send System Evaluation 
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Figure 11-9. VFCT Receive System Evaluation 
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